Researchers spent five years developing an AI-based model to protect freshwater fish worldwide from extinction, with a particular focus on identifying threats to fish before they become endangered. "People sometimes go in to protect species when it's already too late," said Ivan Arismendi, an associate professor in Oregon State University's Department of Fisheries, Wildlife, and Conservation Sciences. "With our model, decision makers can deploy resources in advance before a species becomes imperiled." The findings were recently published in the journal Nature Communications. Nearly one-third of freshwater fish species face possible extinction, threatening food supplies, ecosystems and outdoor recreation.

Open-vocabulary models are a promising new paradigm for image classification. Unlike traditional classification models, open-vocabulary models classify among any arbitrary set of categories specified with natural language during inference. This natural language, called "prompts", typically consists of a set of hand-written templates (e.g., "a photo of a {}") which are completed with each of the category names. This work introduces a simple method to generate higher accuracy prompts, without relying on any explicit knowledge of the task domain and with far fewer hand-constructed sentences. To achieve this, we combine open-vocabulary models with large language models (LLMs) to create Customized Prompts via Language models (CuPL, pronounced "couple"). In particular, we leverage the knowledge contained in LLMs in order to generate many descriptive sentences that contain important discriminating characteristics of the image categories. This allows the model to place a greater importance on these regions in the image when making predictions. We find that this straightforward and general approach improves accuracy on a range of zero-shot image classification benchmarks, including over one percentage point gain on ImageNet. Finally, this simple baseline requires no additional training and remains completely zero-shot. Code available at https://github.com/sarahpratt/CuPL.